The present disclosure relates generally to artificial intelligence and intelligence analysis, and more particularly to an evidence evaluation system and method based on question answering.
Intelligence analysts are given an initial collection of documents, and a set of questions and hypotheses, plus access to additional sources. Their task is to answer the question and provide arguments for and against the hypotheses. After some preliminary work, one or more of them begin to work on a document. Currently, intelligence support software is driven by extracted patterns. Current systems rely on extraction of specific patterns, such as “Person-type travel-type from A-location to B-location”, which can be viewed as a logical form (LF) of the information of interest. Thus, existing systems classify text strings as to whether they contain such patterns of interest, and if so, the existing systems extract the relations of interest, and put them into relational databases for further processing (e.g. data mining, or querying).
Based on such extract patterns, hypothesis formed in intelligence analysis may be confirmed or refuted. However, given the huge amount of data, it is difficult to know if the absence of evidence against a hypothesis is genuine or simply a failure of search. Further, a disadvantage of such approach is that no systems may do it reliably for unrestricted domains, and even for restricted domains the specification of extraction patterns is costly. The reason is that the same facts can be expressed in many different ways and are not necessarily contained in one or even adjacent sentences. Another disadvantage of such approach may stem from the need to understand in advance what types of questions can be asked by analysts. Thus, for example, a set of patterns can be created to cover travel between locations, but an analyst might in a particular case care about other information during the covered travel, for example, how and where the ticket for travel was bought, or the details about refueling a car or an airplane.